Jozef Stefan International Postgraduate School

Despite the general acceptance that the success of machine learning algorithms generally depends on data representation, most NLP and text mining systems and techniques still treat words as indices in a vocabulary, devoid of meaning. We believe that discarding semantic and syntactical similarity between words results in the loss of explanatory factors that would contribute to the success of the above mentioned systems and techniques. In text mining, the common representations also ignore word order. In this work we analyze continuous vector representations of words which preserve semantic and syntactic regularities as well as techniques for composing over them which are sensitive to word order. We show how these have been used to improve Language models, NLP, machine translation, information retrieval and text mining systems. Finally, we present the shared tasks used to evaluate the representations and their compositionality and discuss directions for further work.